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Ensuring gains in the learning outcomes of college students has become a major 
concern for Japanese higher education institutions. In recent decades, national and 
public as well as private universities have been forced to embed learning outcomes 
into their curriculum. A number of studies have shown that the learning outcomes of 
students are strongly associated with the quality of pedagogy and student experience. 
This paper aims to understand the association between college experiences and 
degree of learning among academic fields using data collected in nationwide student 
self-report surveys, the JCIRP. The findings suggest that faculty engagement and 
student experience play pivotal roles in the acquisition of knowledge and skills. Based 
on the findings of this study, it is suggested that pedagogical approaches should be 
improved by embedding aspects of interactive teaching and learning in the classroom 
setting and in the whole curriculum. 
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INTRODUCTION 

Today, higher education institutions around the word are required to take increasing responsibility 
for the quality of their education outcomes; quality assurance and accountability issues have 
become common concerns for higher education institutions the world over. Zumeta (2011) 
defines accountability as “responsibility for one’s actions to someone, or to multiple parties, as a 
result of legal, political, financial, personal, or simply moral ties” (p. 132). The general public is 
becoming more involved with higher education institutions, and is critical when it is not satisfied 
with the effectiveness and the perfonnance of the institution. Consequently, demonstrating quality 
and institutional accountability by achieving clearly stated educational outcomes has become 
important (Ewell, 2007). Japanese higher educational institutions are not exempt from these 
modern demands. A reform movement has emerged, as highlighted in new Ministry of Education, 
Culture, Sports, Science and Technology (MEXT) policies. The Central Council for Education 
(CCE) report, Future of Japanese higher education , declared the 21st Century as the age of the 
knowledge-based society and, in such a society, achieving high-quality educational outcomes are 
important for both individuals and the nation (MEXT 2005). Hence, higher education policy in 
Japan has shifted, in this decade, from research-centred to teaching- and learning-centred. 
Another CCE report, published in 2008, was revolutionary in the sense that it confirmed this 
policy shift. The report, Toward the construction of undergraduate education (MEXT, 2008), 
urged Japanese universities to set common learning outcomes for tertiary education students as 
one method for meeting the quality assurance requirements of globalization. Common learning 
outcomes are called “graduate attributes” and include: generic skills, such as communication, 
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quantitative skills, infonnation literacy, logical thinking and problem solving; intercultural- 
multicultural knowledge, human culture, society and the natural world; and social skills, such as 
teamwork, collaboration, and leadership (MEXT, 2008). Since the 2008 CCE report, assessing the 
types of college environment that accelerates student experience and leads to good learning 
outcomes has become a national concern for Japan. 

In recent decades, Japanese Universities have been forced to embed learning outcomes into the 
curriculum of many of their units regardless of academic field. Studies have shown that the 
learning outcomes of students are positively associated with the quality of pedagogy and student 
experience (Pascarella & Terenzini, 2005; Ogata, 2008). Building on such studies, this paper 
explores the association of college experiences with degree of learning among academic fields. 
This study used quantitative methods to analyse data collected in nationwide, self-report surveys 
of students: the Japanese College Student Survey (JCSS) 2010 and the JCIRP Freshman Survey 
(JFS) 2008. The surveys are part of a series administered by the JCIRP (Japanese Cooperative 
Institutional Research Program), which are designed to obtain infonnation about upper and lower 
division students attending Japanese universities. 

DEVELOPMENT OF JAPANESE COOPERATIVE INSTITUTIONAL RESEARCH 

PROGRAM (JCIRP) 

Banta (2004) suggests that there are two kinds of assessments for measuring student learning 
outcomes: direct and indirect assessments. Direct assessment gauges the direct learning outcomes 
of students through tests, essays, portfolios, graduation examinations, graduation research papers 
and standardized tests in both general and discipline-based education. Indirect assessment gauges 
the learning process, using student surveys about learning behaviors, student experiences, self¬ 
perception and satisfaction. When used in tandem, indirect and direct assessments complement 
one another, offering a broad picture of college outcomes (Gonyea, 2005). 

Over the years, much research has been conducted and much debate has ensured concerning the 
efficacy and accuracy of student assessment in countries such as the US and Australia (Coates, 
2010). Many researchers have dedicated their work to measuring, testing and assessing student 
learning outcomes and have developed various tools for gauging the cognitive, educational, and 
affective progress of students. Many questions on the reliability and validity of self-report student 
surveys, often used by the researchers, have arisen (Borden & Young, 2007). Nevertheless, a 
growing body of empirical evidence shows that self-report student surveys are a valuable method 
for assessing learning outcomes and the college experience of students (Anaya, 1999; Kuh et al, 
2001; Gonyea, 2005; Coates, 2010). Consequently, several standardized student surveys to assess 
college outcomes and experience have been designed (Shavelson, 2010). The Cooperative 
Institutional Research Program (CIRP) and National Survey of Student Engagement (NSSE) are 
two self-report surveys widely used in the US (Kuh, 2004; Pascarella & Terenzini, 2005). In 
Japan, however, few well-researched assessment tools have been developed. That is, Japanese 
student surveys are often designed and applied without underpinning theory to aid in their validity 
(Yamada, 2008). This situation differs markedly from that in the US, where both direct and 
indirect assessment tools have been devised using in-depth research and extensive theory. 

Since 2004, we have been developing student survey assessment tools that stress the 
developmental model of affective and behavioral experiences of student life. With the approval of 
HERI (Higher Education Research Institute) at UCLA (University of California Los Angeles), we 
developed a Japanese version of the College Student Survey, called the Japanese College Student 
Survey (JCSS), and the JCIRP Freshman Survey (JFS). These surveys are specialized version of 
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the CIRP (Cooperative Institutional Research Program) and CSS (College Student Survey) 
developed at HERI that features numerous items—developed specifically for Japanese student, 
which gather information about learning behaviors, experiences, values, motivations, and student 
self-assessment. In 2008, we developed the original Japanese Junior College Student Survey 
(JJCSS). The theoretical background for the survey is based on the work of Astin, who, in 1966, 
firs proposed college impact theory, which predicts student outcomes through a consideration of 
multiple factors. Astin (1985, 1993) examines the origins and processes of student change and 
growth throughout their college lives. Many researches have since used college impact theory as a 
framework for gaining a better understanding of student growth through college life (Tinto, 1993; 
Pascarella & Terenzini, 1991, 2005). 

As already noted, at present, in Japan, more than ever, teaching and learning have gained new 
emphases. It is imperative to refine methods of indirect assessment and apply research finds 
improve teaching and learning in Japanese higher education. One of the basic purposes of the 
JCIRP is to become the method for conducting indirect assessment of college impact on student 
growth. A set of qualitative and quantitative methods were used to validate the JCIRP before and 
after application: the qualitative methods included pilot studies with focus groups and expert 
review; the quantitative methods included item response analysis for the questionnaire items and 
reliability analysis. These methods were similar to those used for validation of assessment 
instruments in the US (Kuh, 2004). After conducting the survey in Japanese institutions, our 
research team analysed the data and produced a report for each of the participating institutions. In 
turn, the institutions provided us with feedback on the ways in which we might improve the 
survey and our reporting of results. This process helped us to improve the reliability and validity 
of the surveys. As shown in Table A1 (Appendix), more than 105,000 students from 678 
institutions have participated in the three different surveys conducted by the JCIRP in the eight 
years to 2012. These surveys are carried out on a voluntary basis by each institution. 
Consequently, some participants in the three surveys belong to the same institution; but the 
majority are recruited from different student populations and different institutions each year. Thus, 
JCIRP surveys provide cross-sectional information and are not designed to be longitudinal 
surveys which follow the same respondents over the course of several years. 

LITERATURE REVIEW OF LEARNING OUTCOMES 

Learning outcomes can be measured on the basis of external effects—that is, efficacy outside the 
formal educational system—or by internal effects, also called “college impact”, showing the 
learning outcomes of students as a result of the quality of pedagogy and student experiences. A 
theory of college impact does not concentrate on any individual process of students’ growth; 
rather, it focuses on the contexts in which a student acts and thinks. Institutional structures, 
policies, programs, and services, as well as attitudes, values and the behaviors of others in 
institutional environments, are all aspects of student growth (Pascarella & Terenzini, 2005; Klein, 
Chun, Hamilton, & Shavelson, 2005; Lambert, Terenzini, & Latuca, 2007; Lichtenstein, 
McConnick, Sheppard, & Puma, 2010). Pascarella and Terenzini (2005) have suggested that the 
institutional structure has both an indirect and direct influence on student development; it includes 
the college environment, the quality of student effort and students’ interactions with other 
students and the faculty. 

Several Japanese studies that focus on learning outcomes of college students in Japan. Murasawa 
(2003), Kuzuki (2006) and Ogata (2008) asked what kinds of knowledge and skills do college 
students obtain through college life. They found that obtained knowledge and skills differ 
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depending on academic fields (disciplines). Yamada (2008, 2009) found that upper division 
students (junior and senior years of undergraduate education) had acquired more knowledge, both 
generally and in their academic fields, than lower division students (first-year students). By the 
time students are promoted to upper division, differences grow between the sciences and the arts 
fields. While students in the arts acquire more general knowledge associated with global and 
cultural knowledge, students in the sciences acquire more knowledge of their fields. Further, 
Yamada (2008) found that the degree of satisfaction with the college experience of upper division 
students is higher than that of lower division students. 

Furuta (2010) confirmed that there were clear differences in knowledge and skills attainment, 
depending on academic fields (disciplines). He further suggested that students in the arts (the 
humanities and the social sciences) tended to self-evaluate higher than students in the sciences 
(the natural sciences and engineering) in tenns of obtained knowledge and skills. However, these 
studies did not delineate how the different student experiences among academic fields contribute 
to the degree of perceived obtained knowledge and skills. 

In a study examining the impact of academic fields, Pascarella and Terenzini (2005) found that a 
person’s major course of study had a selective impact on the development of general cognitive 
skills. A student’s cognitive growth was greatest on measures where the content was most 
consistent with that student’s academic field. Other studies affirm the significance of the 
academic field for student development (Coates & Ainley, 2007; Marks & Coates, 2007). 

Although these studies outside Japan have clarified that the association of various student 
experiences in and outside the classroom have positive impacts on learning outcomes, it is 
questionable whether the findings obtained can be applied to Japanese students studying in 
specific academic fields. The studies carried out by Furuta (2010), Murasawa (2003), Kuzuki 
(2006), Ogata (2008), and Yamada (2008, 2009) did not show how different pedagogies and 
student experiences in different academic fields affect their learning outcomes. Tanimura (2009, 
2010) focused on how differences of pedagogy and class structure in three academic fields 1 
influence learning hours and learning outcomes. The studies showed that the health science field, 
which is tightly coupled with the qualification system in Japan, is distinctive. 

This study aims to clarify how different pedagogies and student experiences in different academic 
fields across national/public and private institutions affect their learning outcomes. Findings from 
this study shed light on how Japanese universities deal with interactive teaching and learning 
issues in today’s knowledge-based society from not only theoretical but also a practical base. 

RESEARCH OBJECTIVES 

This study is part of a nationwide research effort to understand the association of college 
experiences with the degree of learning among academic fields. The study also examined whether 
learning hours differ between academic fields. The CCE report published in March 2012 (MEXT, 
2012), asserted that average learning hours of students should be eight hours per day (including 
inside and outside class learning hours) but the actual average learning hours of Japanese students 
is 4.6 hours a day. The report also points out that the number of learning hours of students in the 
social sciences field are relatively low compared to those of the natural and the health sciences. 
Overall, policymakers are concerned that Japanese students engage in relatively fewer learning 
hours than US students. 


1 Tanimura classified academic fields into three areas: humanities & social sciences, STEM and health sciences. 
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In recent decades, national and public universities have been forced to embed learning outcomes 
into their curriculum in order to deal with accountability requirements. Private universities, which 
have faced a decrease in student numbers, have voluntarily followed suit. Many Japanese 
universities have introduced a variety of programs, including active learning and other 
experiences. Such programs are expected to contribute to increased knowledge and skills. In a 
changing environment, in which Japanese universities are becoming much more teaching centred, 
it is imperative to update the general profile of Japanese students. Following the approach of a 
nationwide study (MEXT, 2012), we examined the variations in learning hours among the 
different academic fields and asked how college experiences contribute to learning outcomes in 
both public and private universities. This study also explored whether there are differences in the 
experiences of students attending the three different types of universities: national (administered 
by the national government), public (administered by local government), and private institutions. 

The JCIRP, a continuous and standard self-report survey, made it possible to delineate trends and 
changes in student college experiences. Thus, as well as obtaining general trend infonnation about 
student college experiences, the following specific research questions were examined; 

• Are there differences between academic fields and between types of institution in obtained 
knowledge and skills? 

• Are there differences in college experiences among academic fields? 

• What are the possible causes of the differences in obtained knowledge and skills among 
academic fields and between lower and upper division students? 

The detenninants of obtained knowledge and skills were also explored. 


METHOD 

Instrument and research framework 

This study employed a quantitative research design using data obtained from JCSS2010. 
JCSS2010 is a series of JCIRP designed to obtain data from upper division students. It consists of 
36 items and 299 variables, including college experiences, satisfaction, self-evaluation for 
learning outcomes and student background. In order to compare degree of satisfaction between 
lower division students and upper division students, JFS2008 data consisting of 35 items and 207 
variables was also used in this study. 

Multiyear self-reported student data can be used to examine important questions. The arrows 
shown in Figure 1 represent: (A) cohort comparisons; (B & C) longitudinal comparisons; and (C 
& D) cross-sectional comparisons. Thus, interpretation and analysis of results based on a model 
illustrated in Figure 1 can clarify environmental factors that affect learning outcomes of students. 
The data from JFS2008 can be matched to the data of JCSS2010 for the same institution (Figure 1, 
Line B), enabling long-term comparison. Some questionnaire items, such as degree of satisfaction, 
match completely between JFS2008 and JCSS2010. Thus, if there is a difference between 
satisfaction of lower and upper division students in the same academic fields of the same 
universities, it is possible to explore the factors that cause the difference. 


104 



Reiko Yamada 



Figure 1: The JCIRP data set and analysis model 


JCSS2010 and JFS2008, as with other self-report surveys, have validity and reliability limitations. 
Kuh et al. (2001) argues that self-report measures are likely to be valid if employed under five 
conditions: the information is known to respondents; the questions are phrased clearly and 
unambiguously; the questions refer to recent activities; the respondents think the questions merit a 
serious and thoughtful response; and answering the question does not threaten, embarrass, or 
violate the privacy of the respondent or encourage the respondent to respond in socially desirable 
ways. Our JCIRP research group discussed the contents of items and reviewed them for each year. 
After the review process, we eliminated inappropriate items and added necessary items. Thus, 
consistent attempts were made to ensure that JCSS2010 and JFS2008 met the five criteria and 
provided accurate and meaningful infonnation about students’ experiences and learning outcomes. 

The research framework aimed to examine the relationship between learning environment and 
learning outcomes among academic fields, as illustrated in Figure 2. Although input factors, such 
as performance and experiences in high school, directly and indirectly influence learning 
outcomes, this study focused particularly on the relationship of environmental factors: type of 
institutions, academic fields, faculty engagement, student experiences, and affective and cognitive 
outcomes. Thus the framework illustrated by the dashed line in Figure 2 will be examined. 


Institutional Environment 

(Type of Institutions, Curriculum & Program) 



E 

Figure 2: Research framework 
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RESULTS 

Profile of Respondents 

Total respondents of the JCSS2010 survey were 8,300 from 82 institutions (17 national, 6 public 
and 59 private universities). Respondents from national and public universities accounted for 28.5 
per cent of respondent and those from private institutions 71.5 per cent. Since these proportion 
align with those of the School Basic Survey, which is a national survey that MEXT requires 
students to complete every year, the JCSS2010 data appeared to represent the general profde of 
Japanese college students. Academic fields included were: 15.5 per cent, humanities; 41.5 per 
cent, social science; 16.1 per cent, natural sciences and engineering; 23.8 per cent, health 
sciences; and 3.1 per cent, others. The proportion of female respondents, at 56.8 per cent, was 
higher than that of male students but, based on the School Basic Survey (MEXT, 2012), the 
proportion of female students who actually attend university for four years was 41.7 per cent. 
Female respondents, therefore, are over-represented in the survey sample. We assumed that this 
gender imbalance in the survey sample is, in part, because most respondents in the health sciences 
are female students of nursing. 

Figure 3 shows that there are differences in number of learning hours outside class time by 
academic fields. As described above, the issue of student’s learning hours outside class time has 
become the subject of heated discussion. Japanese universities employ a credit system and, 
usually, 124-128 credits are required to obtain a bachelor’s degree after four years at university. 
The University Establishment Standard requires a total of 45 hours of learning; that is, students 
are required to study 15 hours in class and 30 hours outside class per one credit per semester. As 
noted by Kaneko (2013), students are obliged to study eight hours a day inside and outside class 
except on Sundays. It should also be noted that, although the JCSS2010 includes questions 
regarding learning hours inside and outside of class, it does not include any additional items 
regarding the content of this learning. Thus, there is no information about what, specifically, 
students learn outside of class. However, as is evident from Figure 3, the learning hours of 
Japanese students falls short of the University Establishment Standards. Students in the health 
sciences, and natural sciences and engineering attend more classes than students in the humanities 
and social sciences (MEXT 2012). Differences in in-class learning hours between academic fields, 
therefore, exists. Of greater concern to MEXT, however, are the differences among academic 
fields in learning hours outside of class. 
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Figure 3: Learning hours outside class by academic fields 
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As the report by the CCE (MEXT, 2012) points out, students in the medical sciences, natural 
sciences and engineering tended to study longer hours than those in the humanities and the social 
sciences. The question emerges whether the differences in learning hours among academic fields 
lead to differences in learning outcomes. It should be understood that learning hours is only one 
variable that affects the learning outcomes of students. 

The dependent variables in this study are the learning outcomes of Japanese undergraduate 
students in three areas: interpersonal skills, classic knowledge and skills, contemporary 
knowledge and skills. Similar to the method used in earlier studies by Pascarella and Terenzini 
(2005), we examined the association of college environment and learning outcomes. Learning 
outcome items were made up of 20 items scored on a 5-point Likert scale related to how much 
students’ learning outcomes improved over the course of the students’ undergraduate education. 
Using factor analysis, we extracted four factors with factor loadings greater than 0.400. Principal 
axis factoring using the varimax method resulted in a 56.6 per cent cumulative contribution. Four 
factors were identified from 19 items. We labelled the factors: interpersonal skills, including the 
items “skill to build human relation (.840),” “skill to carry things out in cooperation with other 
people (.784),” “communication skill (.762),” “leadership ability (.640)”, and “skill for time 
management (.505)”; classic knowledge and skills, comprising “knowledge of particular academic 
fields (.731),” “analytical and problem-solving skills (.669),” “general knowledge (.619),” 
“critical thinking skill (.547),” “IT skill (.500),” “writing skill (.419),” and “oral presentation skill 
(.418)”; contemporary knowledge and skills comprise “knowledge of people of different cultures 
(.695),” “skill to cooperate with people of different cultures (.691)”, “understanding of issues your 
community faces (.683),” and “understanding of issues your nation faces (.655)”; basic skills 
comprise “mathematical skill (.707)” and “foreign language skill (.673).” The Alpha reliabilities 
of these 4 scales are .80, .76, .80 and .72 respectively, with all four over .72. 

Based on previous research findings, the independent variables of this study were categorized as 
control variables (i.e. student input variables) and process variables. Control variables included 
gender, institutional type and academic field. 2 Process variables included students’ experiences 
related to learning inside and outside class, active learning and their adjustment to university 
education. Also included were variables related to faculty engagement in students’ activities and 
students’ satisfaction regarding university education. 

Gender was included as a dummy variable taking a value of 0 for male and 1 for female. 
Institutional type was also a dummy, coded 0 for private universities and 1 for national and public 
universities. Academic fields were categorized into four groups: the humanities, the social 
sciences, the natural sciences and engineering, and the health sciences. The health sciences were 
treated as a reference field group. A humanities variable was dummy coded with academic fields 
in the humanities taking on a value of 1 and all other fields taking on a value of 0. Academic 
fields in the social sciences, and the natural sciences and engineering were similarly dummy 
coded, with academics fields in a category receiving a value of 1 and all other fields receiving a 
value of 0. Descriptive statistics for the dependent and independent variables used in this study 
are presented in Table A2 of the appendix. 


2 The variable, institutional type, is divided into: national & public universities, and private universities. National 
universities (national university corporations) and public universities (established by local prefectural governments) 
are collapsed into one code: national & public universities. Academic fields are divided into four fields: Humanities, 
Social Sciences, Natural Sciences & Engineering, and Health Sciences. Humanities include literature, religious 
studies, history, languages etc. Social sciences include economics, political sciences, sociology, education, policy 
studies and psychology. Natural Sciences and Engineering include natural sciences, biology, mathematics, 
technology and agriculture. Health Sciences include medicine, nursing, and other health services. 
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Of the three academic fields, students in the humanities had the highest scores on learning 
outcomes for interpersonal skills, classic knowledge and skills, and contemporary knowledge. 
While students in the natural sciences and engineering scored high for basic skills, their self- 
evaluation scores on interpersonal skills and contemporary knowledge and skills are the lowest of 
the four academic fields (Table 1). We assume that the sciences and engineering curriculums are 
more systematically structured than those of the social sciences and the humanities; science and 
engineering students must deal with theories, concepts and study within laboratories. The highest 
score for basic skills achieved by the students in the sciences and engineering reflects the 
systematic structure of the curriculum and the teacher-centred approach of the field. This finding 
is in line with those of Malek et al. (2012) who found that students in the natural sciences are 
exposed to more abstract concepts, such as theories, and, thus, were more likely to experience 
teacher-centred approaches. While there is insufficient evidence to demonstrate a positive 
association between high scores in basic skills and a teacher-centred approach, it is clear that the 
teacher-centred approach is utilized more frequently in the so-called hard sciences to teach 
various concepts and theories. 


Table 1: Mean scores of learning outcomes and ANOVA by academic fields 



Humanities 

Social 

Sciences 

STEM 

Health 

Sciences 

Maximum 

DF 

F 


Mean 

SD 

Mean 

SD 

Mean 

SD 

Mean 

SD 

SkOl c 



Interpersonal Skills 

18.39 

2.93 

18.04 

3 

17.89 

3.1 

18.04 

2.9 

25 

3 

6.27* 

Classical Knowledge and 

Skills 

26.5 

3.1 

25.9 

3.5 

26.4 

3.3 

25.6 

3.4 

35 

3 

25.2* 

Contemporary Knowledge 
and Skills 

17.67 

2.7 

17.24 

2.6 

16.76 

2.5 

16.48 

2.5 

25 

3 

63.4* 

Basic Skills 

6.2 

1.4 

6.1 

1.4 

6.7 

1.4 

6.1 

13 

10 

3 

65.4* 


*p<.001 

The curricula of the humanities and the social sciences have more flexibility, and an interactive 
approach to learning can be observed in class. We assume that differences in classroom settings 
explained differences in knowledge and skills obtained. 

The results of a two-way ANOVA for three factors of learning outcomes (Table 2) showed 
significant interaction effects between institutional type and academic fields for obtaining 
interpersonal skills (F=3.243 p<.05) and classic knowledge and skills (F=22.99 p<.001). 
Regarding the acquisition of interpersonal skills, while students of the humanities and the social 
sciences at national and public universities score higher than those at private universities, students 
of the natural sciences, engineering and the health sciences at private universities score higher 
than those at national at public universities. Similarly, in terms of obtaining classic knowledge 
and skills, humanities and the social sciences students at national and public universities had 
higher skills than students at private universities; whereas students of the natural sciences, 
engineering and the health sciences at private universities had higher scores than those at national 
and public universities. The higher score of students of the natural sciences, engineering and the 
health sciences at private universities might be explained by the fact that, in the past, national and 
public universities received more financial assistance from national or local government for 
enhancing facilities and equipment, and private universities did not. Thus, to compete, natural 
science and engineering fields at private universities, which were often established later than 
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those of the public and national universities—and with fewer resources, concentrated on 
improving quality of teaching. 


Table 2: Results of two-way ANOVA for three learning outcomes 


Academic 

fields 


Humanities 

Social Sciences 

Natural Sciences 
and Engineering 

Health Sciences 



National 

and 

Public 

Private 

National 

and 

Public 

Private 

National 

and 

Public 

Private 

National 

and 

Public 

Private 

Interpersonal 

Mean 

18.71 

18.33 

18.19 

17.97 

17.71 

18.05 

18.02 

18.05 

Skills 

SD 

2.814 

2.951 

2.909 

3.043 

3.153 

2.98 

2.832 

2.911 

Classic 

Mean 

27.7 

26.3 

26.72 

25.55 

26.3 

26.46 

25.26 

25.64 

Knowledge 
and Skills 

SD 

2.801 

3.143 

3.208 

3.499 

3.196 

3.44 

3.61 

3.289 

Contemporary 

Mean 

18.44 

17.54 

17.5 

17.12 

16.95 

16.58 

16.66 

16.44 

Knowledge 
and Skills 

SD 

2.369 

2.754 

2.51 

2.663 

2.478 

2.516 

2.654 

2.453 




Main effect 





Institutional tvoc 

Academic field 

Interaction 


Mean 

P 

Mean 

P 

Mean 

P 

Interpersonal Skills 

0.408 


6.492 

*** 

3.243 

* 

Classic Knowledge and Skills 

24.689 

*** 

31.094 

*** 

22.99 

*** 

Contemporary Knowledge and Skills 

35.303 

*** 

53.373 

*** 

2.409 



*p<.05, ***p<.001 


Regarding differences in student classroom experience or in engagement with faculty between 
academic fields, Table 3 shows how often a student-centred pedagogical approach is used in the 
classroom setting. Since the maximum score of each item is 5, a student-centred pedagogical 
approach appeared not to be frequently used in the classroom for any of the academic fields; but 
there are differences. Students in the humanities and the health sciences tended to critically 
review their own literature and materials more often than students in the social sciences, natural 
sciences and engineering. Students in the humanities scored highest in presenting their own ideas 
and research than students in other academic fields. The reason might be that class sizes in the 
humanities fields are relatively small and, thus, more interactive methods can be used. The scores 
for learning useful knowledge and skills for work were highest for students of health science. The 
reason might be that the curriculum content for the health sciences is closely linked to actual skills 
used in the workplace. Students of the natural sciences and engineering, as well as in the health 
sciences tend to leam experientially through experiment and practice—probably because the 
learning environment of those academic fields introduces more opportunities for practical, hands- 
on learning activities. Teaching assistance was highest in the natural science and engineering 
fields; perhaps because the fields involve many graduate students to help teach undergraduate 
students. Assignments in many academic fields were not returned to students with corrections and 
comments. There was little difference among the three academic fields of humanities, social 
sciences and health sciences for the items: “students discuss with each other during class”, 
“student’s opinions are incorporated in class”, and “students set the themes to be discussed in 
class”. Although these pedagogies appear to cultivate students’ independence, these are rarely 
introduced in Japanese classes. 
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Table 3: The degree of usage of student-centred pedagogy in the class 



Total 

Humanities 

Social 

Sciences 

Natural 
Sciences & 
Engineering 

Health 

Sciences 


Mean 

SD 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Students examine their own 
literature and materials 

2.98 

0.82 

3.14 

0.77 

2.90 

0.81 

2.99 

0.78 

3.04 

0.84 

Students present their ideas 
and research 

2.78 

0.78 

3.04 

0.70 

2.79 

0.76 

2.60 

0.77 

2.76 

0.79 

Students learn experientially 
through experiment and 
practice 

2.67 

0.97 

2.32 

0.95 

2.51 

0.93 

2.92 

0.92 

3.00 

0.95 

Students learn useful 
knowledge and skills for work 

2.65 

0.89 

2.32 

0.82 

2.63 

0.84 

2.33 

0.78 

3.09 

0.88 

Students discuss with each 
other during class 

2.60 

0.82 

2.69 

0.82 

2.65 

0.79 

2.33 

0.78 

2.73 

0.84 

Papers are returned to students 
with corrections and 

2.51 

0.82 

2.50 

0.80 

2.41 

0.83 

2.63 

0.77 

2.57 

0.82 

comments 











Student's opinion is 
incorporated in class 

2.44 

0.79 

2.51 

0.78 

2.49 

0.79 

2.28 

0.76 

2.43 

0.80 

Teaching assistants help in a 
class 

2.30 

0.95 

1.91 

0.90 

2.25 

0.90 

2.89 

0.84 

2.25 

0.95 

Students set the theme to be 

J"__ 

2.17 

0.83 

2.26 

0.86 

2.21 

0.82 

1.95 

0.78 

2.18 

0.84 


discussed in class 


Maximum score is 4 


Results of multiple regression analysis 

We used multiple regression analyses to detennine the best predictors of variables of educational 
effect on learning outcomes. Environmental factors were represented by institutional 
environment: type of institution, curriculum and programs. We observed that faculty often 
encourage students and offer advice. We hypothesized that faculty engagement with students as 
well as their teaching methods contribute to the smooth adjustment of students to college life and 
leads to good learning outcomes. Student experience on learning, both quantitatively and 
qualitatively, was hypothesized to closely relate to learning outcomes. Based on the research 
framework depicted in Figure 2 and, having examined the correlation coefficients, we entered the 
variables into a multiple regression model. Three dependent variables: interpersonal skills, classic 
knowledge and skills, and contemporary knowledge and skills, were selected, based on the result 
of factor analysis of learning outcomes. The Female, Institutional Type and Academic Field 
dummies were utilized as control variables. 

The results, summarised in Tables 4, 5 and 6, indicated that there were several common, 
significant detenninants of the learning outcomes: interpersonal skills, classic knowledge and 
skills, and contemporary knowledge and skills (shown by grey shading in the three tables). Each 
of the predictor variables was found to be significant at the p < .001 level. Among the 
independent control variables, the humanities dummy variable was found to significantly predict 
all three learning outcomes. Combined, these predictors accounted for 26.0 per cent of the 
variability in interpersonal skills outcome, 24.7 per cent of the variability in classic knowledge 
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and skills outcome, and 14.5 per cent of the variability in contemporary knowledge and skill 
outcome. Regression analyses revealed that the environmental factors: “skills to manage your 
time effectively”, “deepen friendship with other students”, “students present their ideas and 
research”, “learning hours outside class in a week” significantly predict all three dependent 
variables. Faculty engagement variables: “help students in improving their academic skills”, and 
“students leam useful knowledge and skills for work” were also found to be predictive of learning 
outcomes. 


Table 4: The results of multiple regression analysis for learning outcome: interpersonal skills 


Variable 

B 

P 

t 

P 

Female dummy 

-0.39 

-0.007 

-0.59 

0.557 

Institutional dummy 

0.026 

0.004 

0.361 

0.718 

Humanities dummy 

0.417 

0.051 

3.979 

0.000 

Social sciences dummy 

0.271 

0.045 

3.454 

0.001 

Natural sciences & engineering dummy 

0.163 

0.01 

1.536 

0.125 

Skill to manage your time effectively 

0.848 

0.234 

21.62 

0.000 

Deepen friendship with other students 

0.841 

0.235 

21.55 

0.000 

Students present their ideas and research 

0.254 

0.066 

4.985 

0.000 

Learning hours outside class in a week 

0.072 

0.041 

3.745 

0.000 

Help students in improving their academic skills 

0.146 

0.045 

3.597 

0.000 

Students learn useful knowledge and skills for work 

0.172 

0.051 

4.116 

0.000 

Students discuss with each other during class 

0.213 

0.059 

4.534 

0.000 

Quiz and paper are required in a class 

0.21 

0.049 

4.483 

0.000 

To become acquainted with faculty 

0.264 

0.075 

6.453 

0.000 

R 2 =.262 Adjusted R 2 =,260 


Table 5: The results of multiple regression analysis for learning outcome: classic knowledge and 
skills 



B 

P 

t 

P 

Female dummy 

-0.475 

-0.07 

6.278 

0.000 

Institutional dummy 

0.388 

0.052 

4.719 

0.000 

Humanities dummy 

1.161 

0.137 

10.513 

0.000 

Social sciences dummy 

0.688 

0.101 

7.678 

0.000 

Natural sciences & engineering dummy 

0.728 

0.081 

6.009 

0.000 

Skill to manage your time effectively 

0.649 

0.158 

14.481 

0.000 

Deepen friendship with other students 

0.284 

0.07 

6.363 

0.000 

Students present their ideas and research 

0.462 

0.106 

7.942 

0.000 

Learning hours outside class in a week 

0.295 

0.149 

13.4 

0.000 

Help students in improving their academic skills 

0.293 

0.08 

6.297 

0.000 

Students learn useful knowledge and skills for work 

0.302 

0.08 

6.336 

0.000 

Student examine their own literatures and materials 

0.236 

0.57 

4.608 

0.000 

Quiz and paper are required in a class 

0.439 

0.088 

7.885 

0.000 

Give academic advice and guidance 

0.174 

0.49 

3.86 

0.000 

Satisfaction for overall quality of education 

0.115 

0.04 

3.818 

0.000 

R 2 =.249 Adjusted R 2 =,247 


Table 6: The results of multiple regression analysis for learning outcome: contemporary knowledge 
and skills 
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B 

P 

t 

Female dummy 

-0.136 

-0.026 

-0.0117 

Institutional dummy 

0.372 

0.064 

5.457 

Humanities dummy 

1.235 

0.172 

12.418 

Social sciences dummy 

0.891 

0.169 

12.618 

Natural sciences & engineering dummy 

0.415 

0.59 

4.14 

Skill to manage your time effectively 

0.32 

0.1 

8.606 

Deepen friendship with other students 

0.186 

0.059 

5.05 

Students present their ideas and research 

0.295 

0.087 

6.137 

Learning hours outside class in a week 

0.105 

0.068 

5.76 

Help students in improving their academic skills 

0.183 

0.065 

4.763 

Students learn useful knowledge and skills for work 

0.176 

0.06 

4.454 

Students discuss with each other during class 

0.229 

0.072 

5.149 

To become acquainted with faculty 

0.25 

0.08 

6.464 

R 2 =. 147 Adjusted R 2 =. 145 


Although the Female dummy and Institutional dummy were not found to be significant, the 
Humanities and the Social Sciences dummy were observed to significantly predict the acquisition 
of interpersonal skills. Environmental factors: “students discuss with each other during class”, “to 
become acquainted with faculty”, and “quiz and paper are required in a class” were also found to 
affect the acquisition of interpersonal skills. 

All control independent variables were found to significantly impact classic knowledge and skills 
acquisition. Male students at national and public universities obtained more classic knowledge 
and skills. In tenns of how academic fields predict these learning outcomes, as opposed to the 
health sciences as a reference group, the humanities, social sciences and natural sciences and 
engineering academic fields were found to be positively associated with classic knowledge and 
skills acquisition. In addition to the common student experience variables mentioned above, the 
student experience variable, “students examine their own literatures and materials” was also 
found to be predictive of classic knowledge and skills acquisition. These results indicated that 
student experience variables predict the acquisition of classic knowledge and skills. The faculty 
engagement variables: “quiz and paper are required in a class” and “give academic advising and 
guidance”, and “satisfaction with overall quality of education” also predicted the acquisition of 
classic knowledge and skills. 

Institution and academic field variables significantly predict acquisition of contemporary 
knowledge and skills. Students in the humanities, the social sciences and natural sciences and 
engineering at national and public universities were found to acquire more contemporary 
knowledge and skills than those of the health sciences. The student experience variables: “students 
discuss with each other during class”, “to become acquainted with faculty” all affected the 
acquisition of contemporary knowledge and skills. 

The observation that academic field significantly predicts all three learning outcomes supports our 
hypothesis that acquisition of knowledge and skills differs among academic fields. 

Variables related to student experiences were strong predictors for the acquisition of all learning 
outcomes. The findings of the regression analysis confirmed that longer hours of learning and 
active experiences yielded a higher score on learning outcomes. 
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THE DIFFERENCE BETWEEN JFS2008 AND JCSS2010 

Finally, we examined the question of whether there is a difference between lower and upper 
division students in cross-sectional academic fields by using JFS2008 and JCSS2010 data. The 
number of students with the same academic field and university who participated in both JFS2008 
and JCSS2010 was 3,482 (first year students) and 2,142 (junior and senior). Both surveys 
included the same comprehensive items of degree of satisfaction. The faculty- and course-related 
items were extracted for analysis. 

Although the scope of inference of the survey results was limited, upper division students 
appeared to be more satisfied with more items than lower division students. Across all academic 
fields, upper division students showed lower satisfaction for two items, “relevance of course 
content to daily life”, and “general education” than lower division students. This result is in line 
with previous study results that showed the existence of “relative effectiveness of upper division” 
(Yamada, 2009). However, some differences among academic fields were observed. Forty per 
cent of both lower- and upper-division health sciences students showed a higher degree of 
satisfaction for the question asking “effectiveness of course content for career plan”. But the 
degrees of satisfaction for the other four questions asked of upper division students were not as 
high as that of students of other academic fields. While 40 per cent of lower division students 
were satisfied with general education, only 25 per cent of upper division students indicated 
satisfaction with their general education. Students in the humanities, and the natural sciences and 
engineering students showed higher degrees of satisfaction with general education. 

These results delineate the learning environment around the health sciences: the curriculum and 
programs of the health sciences are strongly influenced by occupational qualification 
requirements, hence, there is less flexibility for students to select courses voluntarily. On the other 
hand, students in the humanities are able to select courses with flexibility; and in the upper 
division, they tend to choose courses that really interest them. Strong motivation for taking a 
course appears to result in a higher degree of satisfaction. This result is in line with Porter’s 
(2006) finding identifying those institutional structures, including institutional density and 
differentiation in the curriculum, that influence student experiences. 


Table 7. The degree of satisfaction for the selected items 



Humanities 

Social 

Sciences 

Natural 
Sciences & 
Engineering 

Health 

Sciences 

All academic 
fields* 


JFS2 

JCSS 

JFS2 

JCSS 

JFS2 

JCSS 

JFS2 

JCSS 

JFS2 

JCSS 


008 

2010 

008 

2010 

008 

2010 

008 

2010 

008 

2010 

Opportunity to talk with 
faculty 

33.8 

32.8 

24.1 

27.5 

22.2 

26.6 

24.7 

27.4 

26 

31.1 

The number of students per 
class 

32.3 

39.3 

26.4 

30.2 

24.8 

34.1 

24.2 

34.1 

27.7 

34.8 

Relevance of course content 
to daily life 

33.1 

28.9 

29.6 

28.5 

23.1 

20.7 

33.4 

24.4 

29 

26.8 

Effectiveness of course 
content for career plan 

23.1 

31.9 

29.3 

30.7 

22 

22.8 

41.6 

38.9 

28.5 

31 

Quality of overall education 

36.6 

44.4 

29.1 

32.9 

26.7 

33.4 

33 

32 

30.7 

36 

General education 

46.8 

50.1 

42.9 

38.2 

37.3 

38.8 

39.5 

24 

41.4 

38.9 


proportion of satisfied + very satisfied 

*all academic fields include other areas such as performing arts and home economics 
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DISCUSSION AND CONCLUSION 

■5 

In addition to the previously discussed items, six items, scored on a 4-point Likert scale, were 
included in the survey to gain information about adjustment to the college environment. Students 
in all academic fields scored the item of “deepening friendship with other students” highest. One 
of the key elements of adjustment in the college environment in every academic field is to get 
along with other students. Students in all academic fields scored the item “to become acquainted 
with faculty” second. Based on these two answers, human relationships and networking are 
regarded as pivotal elements for students in adjusting to the college environment. A small 
difference was observed between students in the hard sciences (natural sciences and engineering, 
and health sciences) and students of the social sciences and humanities regarding the items as 
“obtaining effective time management skills” and “learning strategy skills”. 

Faculty engagement in student learning was also explored. Eleven items focussing on faculty 
engagement, scored on a 4-point Likert scale, were included in the survey. The items related to 
how much faculty was engaged in or helped students improve their skills and adjust to 
undergraduate education. 3 4 The data shows that almost 50 per cent of the faculties in all academic 
fields give advice on student learning in their classes and courses as well as help students to 
improve their academic skills, though the faculties of the natural sciences and engineering are 
more frequently involved in overall student learning improvement activities than those of other 
academic fields. While the percentage proportion of providing an opportunity for participation in 
a research project was less than 20 per cent for the three academic fields, 35.9 per cent of faculties 
of natural sciences and engineering provide an opportunity for students to participate in a research 
project. Also, 45.9 per cent of the hard science faculties help students in achieving their 
professional goals, compared with 37 per cent of the other three academic fields. 

This study aimed to clarify the effects of the university environment on learning outcomes. The 
results supported previous findings that there is an association between institutional environment 
and learning outcomes, as illustrated in the research framework (Figure 1). Further, the results 
provide a new insight, showing that some differences in obtained learning outcomes are due to 
curriculum structure and the pedagogical approaches used in each academic field and institutional 
type. More specialized as well as systematic programs in the natural sciences and in engineering 
require more technical tuition, and hence faculties tend to be more engaged in helping students’ 
learning. However, the result of the effects of college experience on learning outcomes indicates 
that there are commonalities of the effects for three types of knowledge and skills. Students’ 
experiences and faculty engagement appear to play pivotal roles in their acquisition. Several 
implications can be drawn from this result, one of which is that the pedagogical approach should 
be improved by embedding aspects of interactive teaching and learning in the classroom setting 
and in the whole curriculum. The results indicate that improving the pedagogical approach could 
lead to improvements in both quantitative and qualitative student experiences in learning. 


3 1 = much weaker to 4 = much stronger. 

4 These 11 items are: “encourage to pursue graduate/professional degree”, “providing an opportunity in a research 
project”, “give emotional support and encouragement”, “write a letter of recommendation”, “provide feedback 
about your academic work”, “give intellectual challenge and stimulation”, “provide an opportunity to discuss 
coursework outside of class”, “help in achieving your professional goals”, “give academic advising and guidance 
to students”, and “help students in improving their academic skills”. 
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An interesting result from the panel data analysis of 2008 and 2010 was that that degree of 
satisfaction for quality of education and other learning environments differs among academic 
fields. Students of the humanities, in which the curriculum is more flexibly structured, were more 
satisfied than students in the natural sciences, engineering and the health sciences, in which the 
curriculum is systematically structured. This result suggests that self-motivated engagement may 
be more effective than forced-engagement. The result of a greater satisfaction among upper 
division students than lower division students is identical to that found in the Yamada’s (2008) 
earlier study. However, more empirical research should be conducted to clarify whether self- 
motivated engagement of upper division students brings about greater satisfaction. 

Japanese universities have recently endeavoured to improve pedagogy and curricula for proactive 
learning of students. Continuing the accumulation of longitudinal, self-report data of students is 
necessary in order to study how the college environment accelerates the quality and quantity of 
student learning, and leads to improved learning outcomes. Through a better understanding of 
how and what students learn, Japanese universities can develop new pedagogical approaches and 
curricula. 
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APPENDIX 


Table Al. Number of JCIRP participants 



Number of 
Universities 
Participating 
JCSS 

Number of 
Students 
Participating 
JCSS 

Number of 
Universities 
Participating 
JFS 

Number of 
Students 
Participating 
JFS 

Number of 
Universities 
Participating 
JJCSS 

Number of 
Students 
Participating 
JJCSS 

2004 

14 

1491 





2005 

8 

3961 





2007 

16 

6512 





2008 

N/A 

N/A 

163 

19661 

9 

1966 

2009 

24 

4183 

69 

8534 

30 

7244 

2010 

82 

8300 

N/A 

N/A 

23 

7369 

2011 

N/A 

N/A 

119 

10913 

34 

12151 

2012 

58 

5780 

N/A 

N/A 

29 

7102 

Total 

202 

30227 

351 

39108 

125 

35832 


Total number of universities until 2012:678 

Total number of students participation until 2012: 105167 


Table A2. Descriptive statistics for all independent and dependent variables 


Independent variables 

Number 

Mean 

SD 

| Student input variables (control variables) 

Female dummy 

8157 

0.57 

0.50 

Institutional dummy 

8300 

0.29 

0.45 

Humanities dummy 

8001 

0.15 

0.36 

Social Sciences dummy 

8001 

0.41 

0.49 

Natural Sciences & Engineering dummy 

8001 

0.16 

0.37 

Health Sciences as a reference group 

1904 

Process variables 

Student related variables 

Learning hours outside class in a week 

8170 

3.59 

1.69 

Attending hours for class and experiment in a week 

8127 

5.82 

2.04 

Skill to manage your time effectively 

8218 

2.54 

0.82 

To become acquainted with faculty 

8225 

2.68 

0.84 

Deepen friendship with other students 

8229 

3.16 

0.83 

Feel bored in a class 

8221 

3.02 

0.72 

Satisfaction for the quality of overall education 

8221 

3.26 

1.21 

| Faculty related variables 

Students present their ideas and research 

8247 

2.78 


Students discuss with each other during class 

8244 

2.60 

0.82 

Students examine their own literature and materials 

8211 

2.98 

0.82 

Students learn experientially through experiment and practice 

8189 

2.67 

0.97 

Students learn useful knowledge and skills for work 

8215 

2.65 

0.89 

Teaching assistants help in a class 

8204 

2.30 

0.95 

Quiz and paper are required in a class 

8220 

3.26 

0.67 

Give academic advising and guidance to students 

8125 

2.35 

0.94 

Help students in improving their academic skills 

8135 

2.25 

0.92 

| Dependent Variables 

Interpersonal skills 

8061 

18.02 

2.98 

Classic knowledge and skills 

8041 

25.95 

3.39 

Contemporary knowledge and skills 

8056 

17.00 

2.61 


118 
























